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Abstract 

Graph cuts-based algorithms have achieved great suc¬ 
cess in energy minimization for many computer vision ap¬ 
plications. These algorithms provide approximated solu¬ 
tions for multi-label energy functions via move-making ap¬ 
proach. This approach fuses the current solution with a pro¬ 
posal to generate a lower-energy solution. Thus, generating 
the appropriate proposals is necessary for the success of the 
move-making approach. However, not much research ef¬ 
forts has been done on the generation of ''good” proposals, 
especially for non-metric energy functions. In this paper, 
we propose an application-independent and energy-based 
approach to generate "good” proposals. With these pro¬ 
posals, we present a graph cuts-based move-making algo¬ 
rithm called GA-fusion (fusion with graph approximation- 
based proposals). Extensive experiments support that our 
proposal generation is effective across different classes of 
energy functions. The proposed algorithm outperforms oth¬ 
ers both on real and synthetic problems. 

1. Introduction 

Markov random field (MRF) has been used for numerous 
areas in computer vision 1^ . MRFs are generally formu¬ 
lated as follows. Given a graph G = (V,f), the energy 
function of the pairwise MRF is given by 

F/(x) = ^ ^ ^pjXp) A ^ ^ Opq(^Xp^Xq)^ (1) 

pev {p,q)^£ 

where V is the set of nodes, £ is the set of edges, Xp G 
{1, 2, • • • , I/} is the label assigned on node p, and A is the 
weight factor between unary and pairwise terms. Optimiza¬ 
tion of the MRF model is challenging because finding the 
global minimum of the energy function ([T]) is NP-hard in 
general cases. 

There have been numerous researches on optimizing 
aforementioned function. Although they have been suc¬ 
cessful for many different applications, they still end up 
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Figure 1. The basic idea of the overall algorithm. The original 
function is approximated via graph approximation. The approxi¬ 
mated function is optimized, and the solution is used as a proposal 
for the original problem. 


with unsatisfactory solutions when it comes to extremely 
difficult problems. In those kind of problems, many graph 
cuts-based algorithms cannot label sufficient number of 
nodes due to the strong non-submodularity and dual de¬ 
composition cannot decrease gaps due to many frustrated 
cycles 1241 . In this paper, we address this problem by in¬ 
troducing simple graph cuts-based algorithm with the right 
choice of proposal generation scheme. 

Graph cuts-based algorithms have attracted much atten¬ 
tion as an optimization method for MRFs ifT^lbllTlI^fTSl. 
Graph cuts can obtain the exact solution in polynomial time 
when the energy function ([T]) is submodular O. Even if 
the function is not submodular, a partial solution can be ob¬ 
tained with unlabeled nodes using roof duality (QPBO) M 
[22l. Graph cuts have also been used to solve multi-label en¬ 
ergy functions. For this purpose, move-making algorithms 
have been proposed (Tl, in which graph cuts optimize a se¬ 
quence of binary functions to make moves. 

In a move-making algorithm, the most important deci¬ 
sion is the choice of appropriate move-spaces. For exam¬ 
ple, in a-expansiorQ, move-spaces are determined by the 
selected a value. Simple a-expansion strategy has obtained 
satisfactory results when the energy function is metric. Re- 

Tn this paper, a-expansion always refers to QPBO-based a-expansion 
unless noted otherwise. 















cently, a-expansion has been shown to improve when the 
proper order of move-space a is selected instead of iterat¬ 
ing a pre-specified order (H. 

However, ^-expansion does not work well when the en¬ 
ergy function is non-metric. In such a case, reduced bi¬ 
nary problems are no longer submodular. Performance is 
severely degraded when QPBO leaves a considerable num¬ 
ber of unlabeled nodes. To solve this challenge, we need 
more elaborate proposals rather than considering homoge¬ 
neous proposals as in ^-expansion. Fusion move CD can 
be applied to consider general proposals. 

For the success of fusion algorithm, generating appro¬ 
priate proposals is necessary. Although there has been a 
demand for a generic method of proposal generation da, 
little research has been done on the mechanism of “good” 
proposal generation (we will specify the notion of “good” 
proposals in the next section). Instead, most research on 
proposal generation is often limited to application-specific 
approaches EH [13. 

In this paper, we propose a generic and application- 
independent approach to generate “good” proposals for 
non-submodular energy functions. With these propos¬ 
als, we present a graph cuts-based move-making algorithm 
called GA-fusion (fusion with graph approximation-based 
proposals). This method is simple but powerful. It is appli¬ 
cable to any type of energy functions. The basic idea of our 
algorithm is presented in Figure [T] Sec. [3] and [4] describes 
the algorithm in detail. 

We test our approach both in non-metric and metric 
energy functions, while our main concern is optimizing 
non-metric functions. Sec. [5] demonstrates that the pro¬ 
posed approach significantly outperforms existing algo¬ 
rithms for non-metric functions and competitive with other 
state-of-the-art for metric functions. For the non-metric 
case, our algorithm is applied to image deconvolution and 
texture restoration in which conventional approaches of¬ 
ten fail to obtain viable solutions because of strong non¬ 
submodularity. We also evaluated our algorithm on syn¬ 
thetic problems to show robustness to the various types of 
energy functions. 

2. Background and related works 

2.1. Graph cuts-based move-making algorithm 

Graph cuts-based algorithms have a long history ifT^ |6l 
HElIIll. These algorithms have extended the class of ap¬ 
plicable energy functions from binary to multi-label, from 
metric to non-metric, and from pairwise to higher-order en¬ 
ergies (among these, higher-order energies are not the main 
concern of this paper). 

Graph cuts can obtain the global minimum when the 
energy function ([T]) is submodular. In the binary case, 
a function is submodular if every pairwise term satisfies 
Om + Oil < Ooi -f 6>io, where 6>oo represents 6>pg(0,0). 


Graph cuts have also been successfully applied to multi¬ 
label problems. One of the most popular schemes is a- 
expansion. ^-Expansion reduces optimization tasks into 
minimizing a sequence of binary energy function 

Eb{y) = E{xi,{y)), (2) 

where Ei){y) is the function of a binary vector y G 
{0,and X 5 (y) is defined by 

XbAVp) = (1 - Vp) • EVp'a, (3) 

where XbAVp) element-wise operator for X 5 (y) at 

node p, and denotes the current label assigned on node 
p. The label on node p switches between the current label 
and a according to the value of Pp. In such a case, the bi¬ 
nary function Eb{y) is submodular if the original function 
is metric 0. This condition is relaxed in |[T3 such that the 
binary function Eb{y) is submodular if every pairwise term 
satisfies 

0a,a ^/3,7 ^ ^ 01,7 (4) 

^-Expansion is one of the most acclaimed methodolo¬ 
gies; however, standard ^-expansion is not applicable if the 
energy function does not satisfy the condition (0]). In such 
a case, a sequence of reduced binary functions is no longer 
submodular. We may truncate the pairwise terms ll^ [T1 
to optimize these functions, thereby making every pairwise 
term submodular. This strategy works only when the non- 
submodular part of the energy function is very small. If the 
non-submodular part is not negligible, performance is seri¬ 
ously degraded ies. 

Eor the second option, QPBO-based a-expansion can be 
used. In this approach, QPBO is used to optimize sub¬ 
problems of ^-expansion (i.e., reduced binary functions). 
QPBO gives optimal solutions for submodular binary func¬ 
tions; it is also applicable to non-submodular functions. Eor 
non-submodular functions, however, QPBO leaves a cer¬ 
tain number of unlabeled nodes. Although QPBO-based a- 
expansion is usually considered as a better choice than the 
truncation, it also performs very poorly when the reduced 
binary functions have a strong non-submodularity, which 
creates numerous unlabeled nodes. 

Eor the third option, QPBO-based fusion move can be 
considered US). Eusion move is a generalization of a- 
expansion. It produces binary functions in a way similar 
with ^-expansion (Equation (O). The only difference is the 
operator x^AVp)^ which is defined as follows: 

XbAVp) = (1 - Vp) ■ a:™’’ + Vp ■ (5) 

where x^'^^ is a proposal labeling at node p. The value of 
x'P^^ can be different for each node contrary to the case in 
^-expansion. In this case, the function Eb{y) is not always 
guaranteed to be submodular. 


Table 1. Four types of proposal generation strategies. 
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2.2. Proposals for fusion approach 

When the fusion approach is considered, the immediate 
concern is related to the generation of the proposals. The 
choice of proposals changes move-spaces as well as the 
difficulties of the sub-problems, by changing the number 
of non-submodular terms, which consequently affects the 
qualities of the final solutions. 

Although choosing appropriate proposals is of crucial 
importance, little research has been conducted on generat¬ 
ing good proposals. Previous approaches can be roughly 
divided into two categories: offline and online generation. 
Most existing approaches generate proposals offline {type 
1 in Table [T]). Before the optimization begins, multiple 
number of hypotheses are generated by some heuristics. 
For example, Woodford et al ll^ used approximated dis¬ 
parity maps as proposals for stereo application. Lempit- 
sky et al. insi used the Lucas-Kanade (LK) and the Horn- 
Schunck (HS) methods with various parameter settings for 
optical flow. They do not take the objective energy function 
into account when generating proposals. Also, the number 
of proposals is limited by predetermined parameters. In ad¬ 
dition, those proposals are application-specific and require 
domain knowledge. 

Some other approaches generate proposals in runtime 
{type 2, type 3). Contrary to type 1, the number of the pro¬ 
posals is not limited since they dynamically generate pro¬ 
posals online. In ifT^ . proposals are generated by blur¬ 
ring the current solution and random labeling for denois- 
ing application. However, they do not explicitly concern 
objective energy in proposal generation. And, they are 
also application-specific. Recently, Ishikawa ifTSl proposed 
an application-independent method to generate proposals. 
This method uses gradient descent algorithm on the objec¬ 
tive energy function. Although they are energy-aware and 
can be applied to some cases, it is still limited to differen¬ 
tiable energy functions. Thus, this method cannot be ap¬ 
plied even to the Potts model, which is one of the most pop¬ 
ular prior models. In our understanding, this algorithm is 
only meaningful for ordered labels that represent physical 
quantities. 

We introduce a new type of proposal generation {type 4). 
Proposals dynamically generated online so that the number 
of the proposals is not limited, unlike type 1, which uses 
pre-generated set of proposals. The proposals are generated 
in the energy-aware way so that the energies of proposals 


have obvious correlation with final solution. In addition, it 
is generic and applicable to any class of energy functions. 

Lempitsky et al. pointed out two properties for “good” 
proposals: quality of individual proposal and diversity 
among different proposals HU. In addition, we claim in 
this paper that labeling rate is another important factor in 
measuring the quality of a proposal. 

The three properties for good proposals are summarized 
in follows: 

• Quality Good proposals are close to minimum such 
that proposals can guide the solution to minimum by 
fusion moves. In other words, good proposals have 
low energy. 

• Diversity For the success of the fusion approach, di¬ 
versity among different proposals is required. 

• Labeling rate Good proposals result in high label¬ 
ing rate when they are fused with the current solution. 
In other words, good proposals produce easy-to-solve 
sub-problems. 

Note that these conditions are not always necessary. One 
may think of proposals that do not meet the foregoing con¬ 
ditions, but help to obtain a good solution. However, in 
general, if proposals satisfy these conditions, we can expect 
to obtain a good solution. In Sec. [3 we empirically show 
that our proposal exhibits the above properties. 

3. Proposal generation via graph approxima¬ 
tion 

3.1. Graph approximation 

We approximate the original objective function ([T]) to re¬ 
lieve difficulties from non-submodularity. Our motivation 
comes from the well-known fact that less connectivity of a 
graph makes fewer unlabeled nodes 1221 . 

We exploit graph approximation by edge deletion to ob¬ 
tain an approximated function. This approximation is appli¬ 
cable to any class of energy functions, yet they are simple 
and easy. In graph approximation, a graph G = (V, f) is 
approximated as G' = 

More specifically, we approximate the original graph 
with a random subset £' of edges from the original edge 
set 8. Pairwise terms Opq, where (p, q) G £\£', are dropped 
from the energy formulation O- The approximated func¬ 
tion is given by the following. 

E'{^) = yy^0p(xp)+\ ( 6 ) 

pGV {p,q)^£' 

To achieve three properties for “good proposals” men¬ 
tioned in Sec. 12.21 two conditions are required for an ap¬ 
proximated function E' (x). First, the approximated func¬ 
tion should be easy to solve although the original one E{x.) 





is difficult. In other words, more nodes are labeled when 
we apply simple ^-expansion algorithm. Second, the ap¬ 
proximated function should be similar to the original one. 
In other words, solution x' of the approximated function 
should have low energy in terms of the original function. 
Those characteristics are examined in next section. 

There have been other approaches to approximate the 
original function in restricted structures. Some structure are 
known to be tractable, such as bounded treewidth subgraphs 
(e.g. tree and outer-planar graph) ||28] [161 [27] O . However, 
our approximation is not restricted to any type of special 
structure. 

The inappropriateness of these structured approxima¬ 
tions to our framework can be attributed to two main rea¬ 
sons. First, the approximation with the restricted structures 
requires the deletion of too many edges. For example, tree 
structures only have | V| — 1 edges, and 2-bounded treewidth 
graphs have at most 2|V| — 3 edges. In practice, the num¬ 
ber of edges are usually smaller than 2|V| — 3. It is not a 
desirable scenario particularly for highly connected graphs. 
Second, exact optimization of 2-bounded treewidth graphs 
requires too much time. Several seconds to tens of seconds 
may be needed on the moderate size of graphs typically 
used in computer vision EEl. Therefore, embedding this 
structure to our iterative framework is not appropriate. 

Recently, liTOl proposesd the method which iteratively 
minimizes the approximated function. There are two main 
difference with ours. First, they approximate the energies 
in the principled way so that the approximation is same 
with the original one within the trust region or is an up¬ 
per bound of the original one while ours merely drop the 
randomly chosen edges. Second, by careful approximation, 
they guarantee the solution always decreases the original 
energy while ours allow energy to increase in the interme¬ 
diate step. 

In the experimental section, we investigate the approx¬ 
imation with spanning trees and show that it severely de¬ 
grades the performance. 

3.2. Characteristics of approximated function 

In this section, we experimentally show that the graph 
approximation strategy achieves the two aforementioned 
conditions. Through the approximation, solving the func¬ 
tion becomes easier, and the solution of the approximation 
has low energy in terms of original function. 

We design the following experiments to meet the study 
objectives. First, we build the binary non-submodular 
energy functions on a 30-by-30 grid graph with 4- 
neighborhood structure. Unary and pairwise costs are de¬ 
termined as follows. 
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Figure 2. (a) Labeling rates and (b) relative energies are depicted as 
the graph is approximated with a random subset of edges. Relative 
energies are calculated with the original functions. With approxi¬ 
mation, the labeling rate increases and the relative energy becomes 
lower. 


Opq{Xp,Xq)=Y ifXp Xq, 

ySpq'^pq \t Xp ^ Xq^ 

where kp and ^pq are taken from a uniform distribu¬ 
tion f/(0,1), and Spq is randomly chosen from { —1, -bl}. 
When Spq is -|-1, the corresponding pairwise term is 
metric. To vary the difficulties of the problems, 
we control the unary strength, which is computed as 
mQSinppOp (i) / mQSinp^q^ijOpq {i^j) after conversion into nor¬ 
mal form. Since above energy function is already written 
in normal form, it is easy to set the desired unary strength 
by changing the weight factor A. The unary strength is 
changed from 0.2 to 1.2, with interval of 0.2. For each 
unary strength, 100 random instances of energy function 
were generated. As unary strength decreases, QPBO pro¬ 
duces more unlabeled nodes. Of all nodes, 54.7% are la¬ 
beled with the unary strength of 1.2, and none are labeled 
with the unary strength of 0.2. 

We approximate the foregoing functions by graph ap¬ 
proximation and then optimize them using QPBO. For ap¬ 
proximated functions, more nodes are labeled than the orig¬ 
inal ones. The obtained solutions have low energies in terms 
of original functions. These result^ are summarized in Fig¬ 
ure O When the approximation uses a smaller subset S', 
more nodes are labeled. Those results demonstrate that the 
proposed approximation makes the problem not only easy 
to solve but also similar to the original function. 

4. Overall algorithm 

The basic idea of the overall algorithm is depicted in Fig¬ 
ure dl which illustrates a single iteration of the proposed 
algorithm. Our algorithm first approximates original target 
function and then optimizes it to generate proposals. 

A single iteration of algorithm is composed of two steps: 


0p(0) = 0^ Op{l) = kp^ or 0p(0) = kp^0p(l) = 0, (7) the solution are labeled with zero. 


^Here, relative energy is given by the energy of the solution divided by 
the energy of the labeling with zero for all nodes. The unlabeled nodes in 









Algorithm 1 GA-fusion algorithm 
1: initialize the solution Xcurrent 

2: repeat 

3: < proposal generation > 

4: Xproposal ^ OptimizeGA( Xcurrent) 

5: <fusion> 

61 Xcurrent FUSE( Xcurrent5 ^proposal) 

7: until the algorithm converges. 


Algorithm 2 OptimizeGA(x) 

1: initialize the solution with x 
2: for z = 1 ^ A do 

3: build a binary function Eh for expansion with the label a 

4: p-U(0,l) 

5: approximate Eh by E'h using p x 100 percent of randomly 

chosen edges 
6: X ^ arg miux E'h 

7: end for 
8: return x 


proposal generation and fusion, as presented in Algorithms 
[T]and[2l To generate proposals, we first obtain an approx¬ 
imated function E'{'k) of the original £’(x) with p • 100 
percent of edges. 

Estimation of the optimal p is not an easy task. As shown 
in Figure [21 the minimum changes when the unary strength 
varies. We have tried two extremes to choose the parameter 
p. First, we simply fixed the p value throughout all the iter¬ 
ation. It did not work since the optimal p value changed not 
only for each problem, but also for each iteration. And then, 
we tried to estimate the optimal p value every time. It also 
turned out to be inefficient because it caused too much over¬ 
head in time. Instead of taking one of these two extreme 
approaches, the parameter p is randomly drawn from the 
uniform distribution U(0,1) for each iteration. This sim¬ 
ple idea gives surprisingly good performance in the experi¬ 
ments. 

Having an approximated function we perform K 

iterations of ^-expansion using the current labeling Xcun-ent 
as the initial solution. Solution x' obtained by optimizing 
the approximated function is then fused with Xcurrent- Note 
that, the approximated function E' (x) is not fixed through¬ 
out the entire procedure, but it dynamically changes to give 
diversity to proposals. Farger K tends to produce poor pro¬ 
posals by drift the solution far from the current state while 
too small K tends to produce proposals similar to current 
state. The iteration K is set to be min(5, L) throughout en¬ 
tire experiments. At line 6 in Alg. |21 the minimum of E'^ is 
not always tractable due to the non-submodularity. We ap¬ 
proximate the minimum by QPBO while fixing unlabeled 
node to zero. 



Figure 3. Example input images of deconvolution from (a) ’char¬ 
acters’, (b) ’white chessmen’, and (c) ’black chessmen’ datasets. 

5. Experiments 

5.1. Non-metric function optimization 
5.1.1 Image deconvolution 

Image deconvolution is the recovery of an image from a 
blurry and noisy image ED. Given its high connectivity 
and strong non-submodularity, this problem has been re¬ 
ported as a challenging one 1221 . The difficult nature of 
the problem particularly degrades the performance of graph 
cuts-based algorithms. In the benchmark 1221 . graph cuts- 
based algorithms have achieved the poorest results. How¬ 
ever, we demonstrate in the following that graph cuts-based 
algorithm can be severely improved by the proper choice of 
proposals. 

For experiments, we construct the same MRF model 
used in 1211 . First, the original image (colored with three 
labels) is blurred with 3x3 Gaussian kernel where a = 3. 
The image is again distorted with Gaussian pixel-wise noise 
with <7 = 10. For reconstruction, the MRF model with 5x5 
neighborhood window is constructed. Smoothness is given 
by the Potts model. 

We tested various algorithms on three datasets in Fig¬ 
ure [3] They include ‘characters’ dataset (5 images, 200-by- 
320 pixels), ‘white chessmen’ dataset (6 images, 200-by- 
200 pixels), and ‘black chessmen’ dataset (6 images, 200- 
by-200 pixels jl. We compare GA-fusion with other graph 
cuts-based algorithms. They only differ in the strategies to 
generate proposals: homogeneous labeling (a-expansion), 
random labeling (random-fusion), dynamic programming 
on random spanning tree (ST-fusion), and proposed one 
(GA-fusion). The results imply that it is important to choose 
proper proposals. Note that truncation for non-submodular 
part ||2^ (a-expansion(t)) degrades the performance. 

We also apply other algorithms including belief propaga¬ 
tion (BP) ||26][20l, sequential tree-reweighted message pass¬ 
ing (TRw-s) ED dSl, and max-product linear program¬ 
ming with cycle and triplet (MPFP-C) ||24l. For BP, TRW- 
S, and MPFP-C, we used source codes provided by the au¬ 
thors. 


^ Whole data set will be provided in the supplementary material 
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Figure 5. Energy decrease of each method for the deconvolution of 
the Santa image. Two plots shows the same curves from a single 
experiment, with different scales on the y-axis. 


The results are summarized in Table [2l GA-Fusion al¬ 
ways achieves lowest energy solution. Figure [4] shows 
quantitative results for the Santa image. Only GA-fusion 
achieved a fine result. ^-Expansion converged in 3.51 sec¬ 
onds on average. Other algorithms are iterated for 30 sec¬ 
onds except for MPLP-C, which is iterated for 600 seconds. 

We provide more detailed analysis with the Santa image 
in Figures \5\^ Figure [5] shows the energy decrease over 
time in two difference scale. GA-fusion gives best perfor¬ 
mance among all tested algorithms. It is worthy of notice 
that ST-fusion gives poor performance. Some might ex¬ 
pect better results with ST-fusion because they can achieve 
the optimal solution of the approximated function. How¬ 
ever, tree approximation deletes too many edges 92% 
of edges are deleted). To compare GA-proposal and ST- 
proposal, we generate 100 different approximated graphs 
of the Santa problem using our approach and another 100 
using random spanning tree. We optimize former with a- 
expansion and latter with dynamic programming. The re¬ 
sults are plotted on Figure [6l Interestingly, the plot shows 
a curve rather than spread. Note that tree approximation 
requires ^ 92% of edges to be deleted. 

To figure out why our proposed method outperforms oth¬ 
ers, we provide more analysis while each graph cut-based 
algorithm is running (Fig. [7]). It reports the quality (energy) 
of the proposals and labeling ratio of each algorithm. Ac¬ 
cording to section 12.21 “good” proposals satisfy the three 
conditions: quality, labeling rate, and diversity. First, GA- 
fusion produces the proposals with lower energy. It also 
achieves higher labeling rate than others. Finally, random 
jiggling of the plot implies that GA-fusion has very diverse 
proposals. 

5.1.2 Binary texture restoration 

The aim of binary texture restoration is to reconstruct the 
original texture image from a noisy input. Although this 
problem has binary labels, move-making algorithms need 
to be applied because QPBO often fails and gives almost 
unlabeled solutions. 

The energy function for texture restoration is formulated 
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Figure 6. Original energy is approximated and optimized by two 
different methods (GA and ST). For each method 100 different 
random results are plotted. GA-proposals usually have lower en¬ 
ergy than ST-proposals because random spanning tree approxima¬ 
tion deletes too many edges. 
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Figure 7. Experiment on deconvolution of the Santa image. (Left) 
Quality (energy) of the proposals for each iteration using a log 
scale. (Right) Labeling rate with the proposals for each iteration. 


as the same as in IH. Unary cost is given by Op{xp) = 
—/3/{l^\Ip — Xp\), where Ip is the color of the input image 
at pixel p, and /3 is the unary cost weight. Pairwise costs 
are learned by computing joint histograms from the clean 
texture image. The costs for every edge within window size 
w = 35 are learned first. Second, we choose a subset of 
edges to avoid overfitting. S-\-N of most relevant edges are 
chosen, where S is the number of submodular edges, and N 
is the number of non-submodular edges. Relevance is given 
by the covariance of two nodes. 

In the previous works, the numbers of edges S and N and 
the unary weight /7 were determined by learning. However, 
the search space of the parameters was limited because they 
applied conventional graph cuts and QPBO. In O, conven¬ 
tional graph cuts are used, thus N should be fixed to zero. 
In ifTvl QPBO is used to take account of non-submodular 
edges. However, QPBO gives almost unlabeled solutions 
when N is large and f3 is small. 

To evaluate the capability of our algorithm, we con¬ 
trol the model parameters so that each algorithm is applied 
on four different settings: low-connectivity and high-unary 
weight; low-connectivity and low-unary weight; high- 
connectivity and high-unary weight; and high-connectivity 
and low-unary weight. For low connectivity, we use six 





















































Table 2. Image deconvolution results on four input images. Energies and average error rates are reported. The lowest energy for each case 
is in bold; GA-fusion achieves lowest energy for every image. 



GA-fusion 

ST-fusion 

a-Expansion 

Random-fusion 

BP 

TRW-S 

MPLP-C 

[Mean Energy (x 10^)] 
Characters dataset 

-1.86 

346.05 

26.85 

12.54 

13.74 

19.21 

82.28 

White chessmen dataset 

-0.28 

195.73 

14.21 

5.47 

7.06 

9.43 

39.74 

Black chessmen dataset 

1.87 

468.13 

12.59 

28.63 

25.63 

28.22 

78.99 

[Average Error] 
Characters dataset 

1.61% 

27.61% 

9.47% 

13.73% 

22.86% 

24.33% 

26.88% 

White chessmen dataset 

0.63% 

19.63% 

7.59% 

9.07% 

16.77% 

17.94% 

20.29% 

Black chessmen dataset 

2.33% 

65.90% 

7.71% 

37.42% 

61.84% 

63.74% 

65.79% 



(a) GA-fusion 


(b) ST-fusion 


(c) o-expansion (d) Random-fusion 


(e) BP 
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Figure 4. Image deconvolution results on the Santa image. Proposed GA-fusion algorithm achieves best results, (a-d) Four graph cuts- 
based algorithms obtain significantly different results. It implies that the proper choice of proposal is crucial for the success of the graph 
cut-based algorithm. 











Figure 8. Four examples of Brodatz textures (cropped). 


most relevant edges (S = 3, N = 3) and for high connec¬ 
tivity, we use 14 most relevant edges (S = 7, N = 7). The 
unary weight f3 is chosen to be 5 and 20. 

For the input, we use the Brodatz texture dataset (Fig. [8]), 
which contains different types of textures. Among them, 10 
images are chosen for the purpose of this application. The 
chosen images have repeating patterns, and the size of the 
unit pattern is smaller than the window size (35-by-35). The 
images are resized to 256-by-256 pixels and binarized. Salt 
& pepper noise (70%) is then added. 

The results are summarized in Table [3] Relative en- 
ergiefl are averaged over 10 texture images. When the 
problem is easy (low-connectivity and high-unary weight), 
QPBO is able to produce optimal solutions and all method 
except ST-fusion gives satisfactory low-energy results. 
Overall, GA-fusion consistently achieves low energy while 
others do not. QPBO and a-expansion converged in 2.28 
and 3.44 seconds on average, respectively. All other algo- 

^Relative energy is calculated such that the energy of the best solution 
is 0 and that of zero-labeled solution is 100. 


rithms are iterated for 30 seconds. 

5.2. Metric function optimization: OpenGM bench¬ 

mark 

We also evaluated our algorithm on metric functions. 
Some applications from OpenGM2 benchmark |[T^ are 
chosen: inpainting(n4), color segmentation(n4), and object 
segmentation. They all have Potts model for pairwise terms 
with 4-neighborhood structure. Since their energy functions 
are metric, the optimization is relatively easy compared to 
the previous energy functions. 

The results are summarized in Table [H We report the 
average of final energies of GA-fusion, ST-fusion, and 
random-fusion after running 30 sec as well as other repre¬ 
sentative algorithms including ^-expansion, a/3-swap, BP, 
and TRW-S. Since these energy functions are relatively 
easy, the performance differences are not significant. Al¬ 
though GA-fusion aims to optimize non-metric functions, 
it is competitive to other algorithms and better than other 
heuristics such as ST-fusion and random-fusion. 

5.3. Synthetic function optimization 

We compare our algorithm with others on various types 
of synthetic MRF problems to analyze performance further. 

Four different types of graph structure are utilized: grid 
graphs with 4, 8, and 24 neighbors; and fully connected 
graph. The size of the grid graph is set to 30-by-30 and 
the size of the fully connected graph is 50. For each graph 
























Table 3. Texture restoration experiments on 10 Brodatz textures. Average of relative energies is reported. Four different types of energy 
are considered by changing the number of pairwise costs and unary weight. The lowest energy for each case is in bold. 


Energy type 

QPBO 

GA-fusion 

ST-fusion 

a-expansion 

random-fusion 

BP 

TRW-S 

low-connectivity 
& high-unary weight 

0.0 

0.0 

1.9 

0.1 

0.1 

0.0 

0.0 

low-connectivity 
& low-unary weight 

n/a 

1.5 

25.6 

2.8 

5.1 

3.6 

10.6 

high-connectivity 
& high-unary weight 

n/a 

0.9 

25.3 

0.1 

0.9 

0.4 

3.3 

high-connectivity 
& low-unary weight 

n/a 

2.2 

38.3 

8.3 

4.6 

6.0 

11.6 


Table 4. Mean energies obtained from deferent algorithms on metric energy functions. Test bed is from OpenGM2 benchmark. They all 
use the Potts model for designing pairwise terms. 



GA-fusion 

ST-fusion 

a-Expansion 

a/3-Swap 

Randon-fusion 

BP 

TRW-S 

Inpainting(n4) 

454.35 

466.92 

454.35 

454.75 

545.96 

454.35 

490.48 

Color segmentation(n4) 

20024.23 

20139.12 

20031.81 

20049.9 

24405.14 

20094.03 

20012.18 

Object segmentation 

31323.07 

31883.57 

31317.23 

31323.18 

62834.62 

35775.27 

31317.23 


Table 5. Energies obtained from deferent algorithms on synthetic 
problems. Test bed was designed to evaluate each algorithm on 
different ratios of non-metric term, coupling strengths A, and con¬ 
nectivities. The name of the problem set indicates ''X-(non-metric 
rate)-(graph structure^. For each row, 10 results for different in¬ 
stances are averaged. The lowest energy for each case is in bold. 
GA-fusion consistently finds low energy solutions. 


Energy type 

GA 

ST 

a-Exp 

Rand 

BP 

TRW-S 

1-50-GRID4 

0.1 

0.0 

90.7 

0.3 

5.7 

6.8 

1-50-GRID8 

0.4 

2.0 

100.0 

1.5 

9.0 

17.1 

1-50-GRID24 

1.1 

33.2 

100.0 

0.3 

11.5 

17.2 

1-50-FULL 

0.4 

59.5 

100.0 

45.3 

74.1 

22.4 

10-50-GRID4 

0.2 

0.0 

100.0 

0.9 

11.4 

12.6 

10-50-GRID8 

0.1 

1.0 

100.0 

0.4 

12.0 

13.5 

10-50-GRID24 

3.6 

27.1 

100.0 

0.7 

15.2 

16.5 

10-50-FULL 

0.0 

1.7 

100.0 

5.2 

1.8 

1.8 

1-100-GRID8 

0.3 

0.8 

100.0 

0.0 

1.2 

2.0 

1-100-GRID24 

0.0 

0.9 

100.0 

3.0 

2.6 

2.5 

1-100-FULL 

0.1 

0.7 

100.0 

6.1 

1.9 

1.9 

10-100-GRID8 

0.2 

1.6 

100.0 

0.1 

0.4 

0.4 

10-100-GRID24 

0.0 

2.2 

100.0 

3.1 

2.2 

1.8 

10-100-FULL 

0.9 

41.4 

100.0 

58.3 

74.8 

22.4 


structure, we built five-label problems. Each unary cost 
is assigned by random sampling from uniform distribu¬ 
tion: Op{xp) ^ f/(0,1). Pairwise costs are designed using 
the same method in section 13.21 (Equation (O). The dif¬ 
ficulties of each problem are controlled by changing cou¬ 
pling strength A in the energy function ([T]). The amount 
of non-metric terms are set to 50% and 100% (non-metric 
term means pairwise cost which does not satisfy the con¬ 
dition (lUjl. Ultimately, we construct 14 different types of 
MRE models, which are summarized in Table[5]as ''\-{non- 
metric rate)-(graph structure^. Eor each type, 10 random 
instances of problems are generated. 

^For 4-neighborhood grid graph, 100% of non-metric terms are impos¬ 
sible because by simply flipping labels every term meets the condition (|4j 


Table [5]reports the average of final energy from different 
algorithms. Some algorithms achieve low energy solutions 
with specific type of the energy function. GA-fusion consis¬ 
tently gives low energy solutions throughout all the energy 
type. 

The following are some details on the experimental set¬ 
tings. Graph cut-based algorithms start from the zero- 
labeled initial. Every algorithm, except ^-expansion, is run 
for 10 sec because they do not follow a fixed rule for conver¬ 
gence. The experiment shows that 10 sec is enough time for 
every algorithm to converge. Although a-expansion is fast, 
converging in less than a second, it mostly ended up with 
an zero-label. It is because that reduced sub-problem is too 
difficult and QPBO produces none of the labeled nodes in 
most cases. 

6. Conclusions 

Graph cuts-based algorithm is one of the most acclaimed 
algorithms for optimizing MRE energy functions. They can 
obtain the optimal solution for a submodular binary func¬ 
tion and give a good approximation for multi-label function 
through the move-making approach. In the move-making 
approach, appropriate choice of the move space is crucial 
to performance. In other words, good proposal genera¬ 
tion is required. However, efficient and generic propos¬ 
als have not been available. Most works have relied on 
heuristic and application-specific ways. Thus, the present 
paper proposed a simple and application-independent way 
to generate proposals. With this proposal generation, we 
present a graph cuts-based move-making algorithm called 
GA-fusion, where the proposal is generated from approxi¬ 
mated functions via graph approximation. We tested our al¬ 
gorithm on real and synthetic problems. Our experimental 
results show that our algorithm outperforms other methods, 
particularly when the problems are difficult. 
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